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Abstract 


This study examines the differential effeets on student subgroups of using the ACT® 
College Readiness Assessment Composite (ACTC) score and high school grade point average 
(HSGPA) for making admission decisions. The subgroup characteristics investigated include 
race/ethnicity, gender, and income. For each student subgroup, we examine the effect of using a 
total group cut point for ACTC score, HSGPA, or both to predict first-year college grade point 
average (FYGPA) and the estimated effects of using these predictors to make admission 
decisions. 

The data for the study consisted of over 137,000 first-year entering students from 259 
two- and four-year institutions and over 498,000 nonenrolled students. The present research 
estimates the probability of attainment of a specific level of success in college. We also utilize 
validity statistics at specific cutoff values to evaluate the effects on dichotomous outcomes. 

Across student subgroups, the joint use of ACTC score and HSGPA resulted in greater 
prediction accuracy than when either predictor was used alone. Furthermore, the use of a total- 
group cutoff score for both ACTC score and HSGPA slightly overpredict the probability of 
success of Hispanic and African-American students, males, and lower-income students. Both 
ACTC score and HSGPA slightly underpredict the probability of success of White students, 
females, and higher-income students. These findings suggest, therefore, that African American, 
Hispanic, and lower-income students are not disadvantaged by the use of a total-group cutoff for 
making admission decisions. 
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Differential Effects of Using ACT® College Readiness Assessment Scores and 
High School GPA to Predict First-Year College GPA among Racial/Ethnic, Gender, 

and Income Groups 

Historically postsecondary institutions have been interested in inereasing the aeademie 
preparation and diversity of their ineoming freshman elass (Breland, Maxey, MeLure, Valiga, 
Boatwright, Ganley, & Jenkins, 1995). They are attempting to do this within the eontinually 
ebbing and flowing institutional eontext of soeietal and applieant trends, the values emphasized 
by stakeholders, power to affeet ehange, and legal, fmaneial, or other eonstraints (Bean, 1990). 
Typieally, postseeondary institutions eonsider grades in eollege preparatory eourses, strength of 
eurrieulum, standardized admission test seores, and overall high sehool grade point average 
(HSGPA) along with other non-aeademie measures (Clinedinst, Hurley, & Hawkins, 2011) to 
seleet students to attend their institution. As diseussed by Sawyer (2013), the eriteria admission 
offiees use to define sueeess at their institution may vary depending upon institutional need and 
their mission. Admission offiees are, in praetieal terms, looking for tools that will help them 
eorreetly and equitably identify and admit students into their institutions that are likely to have a 
sueeessful postseeondary eareer at their institution. 

Some authors have eontended that standardized admission tests unfairly disadvantage 
partieular raeial/ethnie, gender, and ineome groups, and that they have weak predietive power 
(Atkinson and Geiser, 2011, FairTest, 2007). Partially as a response to these statements, some 
institutions have adopted alternative admission eriteria. These institutions may require 
standardized test seores if other aeademie eriteria are not met, use standardized test seores for 
eourse plaeement, or may use standardized test seores for eertain applieant populations 
(Milewski & Camara, 2002). Alternatively, institutions may eonsider standardized test seores 
only if applieants ehoose to submit them (Wake Forest University, 2012; Sarah Lawrenee 
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College, 2012) or may allow students to submit a variety of standardized test seores (Hamilton 
College, 2012). 

Regardless of how assessments are used, it is neeessary for developers to eontinually 
doeument their appropriateness for both the population and population subgroups (Ameriean 
Educational Research Association (AERA), 1999). Prior research has addressed some of these 
concerns (Young, 2001). Research related to informing admission decisions should respond to 
perceived bias and assist postsecondary institutions make more informed admission decisions. 
This can be done by examining not only the statistical strength of individual predictors with 
postsecondary outcomes but also the usefulness of and potential differential effects of using pre- 
college measures for making admission decisions. 

Sawyer (2013) investigated the utility of using HSGPA and the ACT Composite (ACTC) 
score to make admission decisions for the purpose of maximizing academic success and 
accurately identifying potentially successful applicants. The results suggested that HSGPA is 
more useful than admission test scores in situations involving low selectivity in admission and 
minimal to average academic performance in college. In contrast, test scores are more useful 
than HSGPA in situations involving high selectivity and high academic performance. In nearly 
all contexts, test scores had incremental usefulness beyond HSGPA. 

Prior research has also investigated the perceptions of bias in standardized testing, as well 
as the advocacy for the use of HSGPA alone for admission decisions. Eor example, research 
conducted by the College Board found that the SAT tends to underpredict EYGPA for females 
and overpredict EYGPA for raciakethnic minority students (Mattem, Patterson, Shaw, Korbin, & 
Barbuti, 2008). They further found that the SAT was more predictive of EYGPA for females 
than males and for White students than other racial/ethnic subgroups. 
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Noble (2003) examined the validity of using ACTC seore, HSGPA, or both for making 
admission deeisions for Afriean Ameriean, Hispanie, and White students. Using hypothetieal 
cutoffs based on optimal predictions of success for making admission decisions, the author 
compared prediction accuracy across subgroups and predictors. When HSGPA was used as the 
sole predictor of FYGPA, African American, Hispanic, and White students with the same 
HSGPA did not have the same probability of attaining a FYGPA of 2.5 or higher. In fact, 
HSGPA overpredicted success for African American and Hispanic students. While a similar 
overprediction occurred when ACTC score was used, the magnitude of the overprediction was 
not as great. Moreover, for the three racial/ethnic groups studied, when ACTC score and 
HSGPA were used in combination, the estimated percentage of students for whom a correct 
admission decision was made was increased relative to using either measure alone. 

Sawyer (1985) examined the differential prediction of HSGPA and ACTC score on 
FYGPA among gender, racial/ethnic, and age groups. This study examined students from the 
graduating classes of 1974, 1975, and 1976. Among other findings, this study found that FYGPA 
for African American and Hispanic students was slightly overpredicted relative to White 
students.^ Additionally, FYGPA for female students was slightly underpredicted relative to male 
students. 

As further explained by Young (2001), it is apparent from previous research that 
subgroup differences do, in fact, occur in validity and prediction. Moreover these differences 
can vary considerably both between and within racial/ethnic subgroups. Young further proposes 
that minority student may experience difficulty in adjusting to postsecondary institutions because 
of Anglo-centric campus environments. In regard to gender differences. Young states that 

' In the graduating classes of 1974, 1975, and 1976 the ACT Student Profile Section designated “Mexican American/Chicano” and “Puerto Rican 
or Spanish-Speaking American.” Sawyer (1985) used students who self-identified as “Mexican American/Chicano.” In the 2010-1 1 academic 
year, race/ethnicity values were changed to comply with federal reporting guidelines. 
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observed differenees may be due to eollege and major selection differences, differential grading 
practices, and gender bias in standardized test score meaning. 

As explained by the College Board (1997), these subgroup differences are not an inherent 
indication of problematic testing programs. Simple geographic and demographic differences 
contribute to these types of population subgroup differences in performance. Additionally, prior 
coursework, aspirations, and differing educational opportunities might contribute to observed 
subgroup differences. The presence of these observed differences may in fact speak more to 
factors unrelated to the testing program being examined than with problematic features of a 
given standardized test. 

The present research provides an updated and more comprehensive investigation of 
differential effects on student subgroups of using ACTC score or HSGPA to predict FYGPA. 
Specifically, the work of Sawyer (2013) is expanded from examining the incremental validity of 
the use of the ACT scores beyond HSGPA for making admission decisions by investigating these 
effects across student subgroups to ensure equity in the admission process. This investigation 
into racial/ethnic and gender differences provides an update to more current data sources and/or 
methodology from Sawyer’s (1985) and Noble’s (2003) studies. Furthermore, the present study 
will take a first look at the differential prediction of ACTC score and HSGPA for making 
admission decisions across income levels. 

This study therefore addresses two fundamental questions for student subgroups 
regarding the use of hypothetical total-group cutoffs for ACTC score and HSGPA to predict 
FYGPA for informing admission decisions: 

1. Do the probabilities of attaining successive levels of FYGPA differ across student 


subgroups? 
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2. Do the estimated effeets of using these predietors to make admission deeisions differ 
aeross student subgroups? 

Although two-year institutions traditionally have open admission polieies that do not 
require minimum levels of standardized test seores or HSGPA, about one-fifth use standardized 
test seores and HSGPA in a eounseling eontext during the admission proeess (Breland, Maxey, 
Gernand, Gumming, & Trapani, 2002). Faeed with eonstraints on their finaneial support, some 
two-year institutions might in the future be required to restriet enrollment to students who are 
likely to eomplete a two-year program or transfer to a four-year institution (Gonzalez, 2012). 
Additionally, two-year institutions are being eneouraged to evaluate intermediate outeomes as a 
way of measuring degree progress and inereasing degree eompletion rates (Moore, Shuloek, & 
Offenstein, 2009). With these faetors in mind, the eurrent study applies the same methodology 
to both two- and four-year institutions in order to evaluate the use of ACTC seore and HSGPA 
for identifying students who are likely to be sueeessful in the first year of eollege. 

Data 

The data for the study ineluded 259 two- and four-year institutions partieipating in 
act’s® Predietion Researeh Serviee or in speeial researeh projeets (Sawyer, 2013). The data 
eonsisted of over 137,000 first-time entering students in the 2003-2004 (< 1%), 2004-2005 
(36%), 2005-2006 (61%), and 2006-2007 (3%) aeademie years.^ FYGPA was provided by the 
institutions themselves. HSGPAs were based on students’ self-report of grades from a possibility 
of 23 high sehool eourses in English, mathematies, Soeial Studies, and Seienee; students 
provided the information at the time they registered for the ACT. 

The present study examined raeial/ethnie, gender, and ineome subgroups. For 
raee/ethnieity. White, Afriean Ameriean, and Hispanie students were investigated. Other raees 


^ Entering freshman class percentages do not sum to 100% due to rounding. 
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were not included in the analysis because of subgroup sample size limitations. For income, 
students were classified as less than $36,000, $36,000 to $60,000, or greater than $60,000. 
Race/ethnicity, gender, and income were provided by students at the time they registered for the 
ACT. 

In order for individual students to be included in the analysis, students must have had at 
least one valid the ACT record in the three years prior to entering a postsecondary institution. A 
minimum subgroup sample size of 10 was required for inclusion of a postsecondary institution. 

Most of the 259 institutions in the sample were four-year public institutions and had a 
small percentage of African American and Hispanic students (see Table 1). There were equal 
percentages of institutions with either traditional or liberal/open admission policies. The majority 
of institutions were from the North Central or Southern accrediting regions. 


^ A minimum sample size restriction was implemented to aid in stabilization of institution-specific model 
coefficients as well as to allow greater intra-institution variability on the student demographic variables being 
investigated. 
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Table 1 

Institution Characteristics 


Characteristic 


Median undergraduate enrollment 

2,403 

Median pereentage Afriean Ameriean and Hispanie 

12 

Median average the ACT Composite seore 

21.5 

Median average HSGPA 

3.3 

Pereent four-year 

74 

Pereent publie 

68 

Seleetivity (in pereent) 

Seleetive/highly seleetive 

17 

Traditional 

37 

LiberaPOpen 

37 

Unknown 

8 

Aeerediting Region (in pereent) 

North Central 

59 

Southern 

32 

Northwestern 

3 

Middle States 

3 

New England 

2 

Western 

1 

Unknown 

1 


Note. Percentages may not sum to 100% due to rounding. Reprinted from the ACT Institutional Data Questionnaire. 


Data for gender and income were not available from the Institutional Data Questionnaire. 


Because it was not possible to construct the true applicant pool for these institutions, an 
approximate pool was developed. This pool included all students from the identified years who 
sent an ACT score report to at least one of the 259 institutions, but did not enroll there, plus the 
students who did enroll there. While it is possible that some of the nonenrolled students did not 
formally apply for admission to the institutions to which they sent their scores, it was not 
possible to distinguish the non-applicants from actual applicants. It is also possible that some 
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applicants to these institutions did not submit an official ACT score report. For the purposes of 
the analyses in this paper seore senders are treated as applieants. 

Mean ACT Composite scores and mean HSGPA values were computed by institution. 
Means were ealeulated for enrolled students, as well as for students in the entire applieant pool. 
Two different FYGPA outeomes were examined in this study: attainment of a 2.5 or higher 
FYGPA or attainment of 3.0 or higher FYGPA. These FYGPA values approximately eorrespond 
to the 30*'^ and 50**' percentiles, respeetively, of observed FYGPA. The percentage of enrolled 
students with a FYGPA of 2.5 or higher and 3.0 or higher was also ealeulated by institution. 
Distributions of these means and pereentages were then summarized aeross institutions using 
minimum, median, and maximum values. Similar analyses were eondueted for eaeh raeial/ethnie, 
gender, and income group. Multiple cohorts of students at an institution were eombined for the 
analyses. 

As was the ease for Noble (2003), the analyses were limited to sueeess levels of 2.5 or 
higher, and 3.0 or higher, FYGPA. The relatively small proportion of students aehieving a 
FYGPA of less than 2.5 at many institutions resulted in eonsiderably fewer institutions 
produeing viable models. The 2.5 or higher and 3.0 or higher sueeess levels were therefore 
seleeted to maximize the number of institutions in both samples for whieh models eould be 
developed. 

The total applieant pool eonsisted of over 137,000 enrolled and over 498,000 nonenrolled 
students. Typieal values aeross institutions for the total applieant sample were similar to those 
for enrolled ACT-tested freshmen nationally between 2004 and 2007 (ACT, 2004, 2005, 2006, 
2007) on median average ACTC seore, pereentage of students taking a eore eurrieulum, and 
pereentages of lower-, middle-, and higher-ineome students (see Table 2). Students in both the 
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enrolled and total applicant samples tended to have higher HSGPAs, on average, than enrolled 
ACT-tested students nationally. While the median average ACTC score for enrolled ACT-tested 
students nationally and that for the total applicant sample were similar, both were lower than that 
of the enrolled students from the study sample. The enrolled sample typically had a greater 
percentage of core -taking students’ and middle- and higher-income students than either the 
applicant sample or enrolled ACT-tested students nationally. Both the enrolled and total 
applicant samples typically had a greater median percentage of male students and a smaller 
median percentage of female students than enrolled ACT-tested students nationally. 
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Table 2 

Summary, across Institutions, of Sample Size and Average Student Characteristics 




Enrolled students 


Applicants 


Enrolled students nationally** 

Characteristic Median 

Minimum 

Maximum 

Median 

Minimum 

Maximum 

Median 

Minimum 

Maximum 

Sample size 

1,311 

14 

5,975 

8,357 

46 

35,537 

1,092 

4 

86,510 

Mean HSGPA 

3.4 

2.0 

3.8 

3.4 

2.6 

3.7 

3.0 

1.8 

3.7 

Mean ACTC 

22.0 

15.0 

28.7 

20.8 

14.0 

26.7 

20.9 

14.9 

29.8 

Mean First year GPA 

2.7 

1.5 

3.6 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Percent African 
American and 
Hispanic 

12 

0 

100 

16 

2 

99 

11 

0 

97 

Percent taking core 
curriculum 

75 

32 

90 

67 

24 

81 

67 

14 

92 

Percent meeting success level 

Percent with 
FYGPA of 2.5 or 
higher 

69 

1 

100 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Percent with 
FYGPA of 3.0 or 
higher 

48 

1 

96 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Race/ethnicity (in percent) 

White 

78 

0 

100 

76 

1 

95 

79 

0 

100 

African American 

6 

0 

96 

8 

0 

92 

6 

0 

93 

Hispanic 

2 

0 

51 

2 

0 

57 

2 

0 

66 

Gender (in percent) 

Male 

57 

21 

97 

54 

18 

96 

42 

0 

100 

Female 

40 

0 

78 

44 

2 

82 

56 

0 

100 

Unknown 

1 

0 

9 

2 

0 

7 

1 

0 

13 

Income (in percent) 

Less than $30,000 

17 

4 

69 

19 

7 

66 

17 

0 

65 

$30,000 to $60,000 

38 

23 

60 

29 

14 

42 

30 

0 

52 

More than $60,000 

43 

6 

66 

33 

5 

49 

31 

0 

100 

Unknown 

0 

0 

0 

18 

11 

41 

19 

0 

62 


*Missing income range was imputed for enrolled students 

**Data for students from national ACT Class Profile Reports (2004, 2005, 2006, and 2007). 
N/A = Data not available. 


Table 3 summarizes the distribution of the eorrelations of the pre-eollege variables and 
FYGPA aeross the 259 institutions in the sample. Typieally, HSGPA had a higher eorrelation 


with FYGPA than ACTC seore. The minimum and maximum eorrelations observed aeross 
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institutions demonstrate significant variation in the correlations at the institutions in the sample. 
At two institutions both ACTC score and HSGPA were negatively correlated with FYGPA, 
however the correlations were not significant. ACTC score and HSGPA typically accounted for 
23% of the variance in FYGPA however there was considerable variation in R values across 
institutions. 

Table 3 

Enrolled Student Correlations among Institutions 


Correlation 

Median 

Minimum 

Maximum 

HSGPA/ACTC 

0.43 

0.07 

0.67 

FYGPA/HSGPA 

0.43 

-0.09 

0.74 

FYGPA/ACTC 

0.36 

-0.09 

0.62 

FYGPA/HSGPA & ACTC multiple R 

0.48 

0.06 

0.78 

FYGPA/HSGPA & ACTC multiple 

0.23 

0.00 

0.60 


Across racial/ethnic, gender, and income subgroups, and institutions, median average 
ACTC scores and HSGPAs for the enrolled sample were generally higher than those for the 
applicant sample (see Table 4). Relative to all enrolled ACT-tested students nationally, students 
in the enrolled sample typically had slightly higher median ACTC scores and HSGPA, on 
average. Students in the applicant sample, on the other hand, tended to have lower median ACTC 
scores and higher HSGPAs, on average, than all enrolled ACT-tested students nationally. 
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Table 4 

Median Student Characteristics across Institutions by Subgroup 





Enrolled stndents 


Applicant pool 

Enrolled 

stndents 

nationally* 



Mean 

Percent with 
FYGPA of 

Mean 

Mean 

Characteristic 

Statistic 

ACTC 

HSGPA 

2.5 or 
higher 

3.0 or 
higher 

ACTC 

HSGPA 

ACTC 

HSGPA 


Med. 

21.9 

3.3 

71 

50 

20.6 

3.3 

21.6 

3.0 

White 

Min. 

17.7 

1.8 

0 

0 

16.9 

2.7 

13.0 

1.8 


Max. 

29.4 

3.8 

100 

99 

26.8 

3.7 

30.6 

3.7 


Med. 

17.7 

3.0 

42 

19 

18.5 

3.1 

17.4 

2.7 

African 


Mm 

13.0 

1.8 

0 

0 

11.3 

2.0 

11.0 

0.0 

American 


Max 

27.0 

4.0 

100 

100 

25.3 

3.9 

27.0 

3.6 


Med. 

20.0 

3.2 

60 

36 

17.8 

3.1 

19.8 

2.9 

Hispanic 

Min 

12.0 

1.9 

0 

0 

10.0 

1.4 

11.0 

0.0 


Max 

27.0 

3.9 

100 

100 

25.3 

3.8 

28.0 

4.0 


Med. 

21.1 

3.4 

71 

52 

20.1 

3.2 

21.0 

3.1 

Female 

Min. 

14.8 

2.6 

1 

1 

14.3 

2.6 

15.4 

0.0 


Max. 

28.7 

3.8 

100 

96 

27.0 

3.7 

29.5 

3.7 


Med. 

21.1 

3.2 

60 

38 

19.2 

3.1 

20.9 

2.8 

Males 

Min. 

15.1 

1.7 

2 

0 

13.3 

2.5 

14.4 

1.8 


Max. 

28.7 

3.8 

100 

95 

0.0 

0.0 

30.2 

4.0 


Med. 

20.0 

3.2 

59 

37 

18.1 

3.1 

19.6 

2.9 

Lower-income 

Min. 

13.4 

2.2 

0 

0 

13.8 

2.5 

14.9 

1.3 



Max. 

26.9 

3.8 

100 

100 

22.8 

3.5 

28.7 

0.3 


Med. 

21.2 

3.3 

67 

46 

20.0 

3.2 

21.0 

3.0 

Middle-income 

Min. 

15.0 

2.0 

1 

0 

14.5 

2.5 

14.1 

1.9 

(SiUK - SbUK) 


Max. 

28.6 

3.8 

100 

94 

25.5 

3.6 

29.4 

0.3 


Med. 

21.7 

3.3 

73 

51 

20.9 

3.3 

21.7 

3.0 

Higher-income 

Min. 

15.1 

1.8 

1 

0 

15.7 

2.6 

14.4 

0.0 

(> SOUKj 


Max. 

29.1 

3.8 

100 

98 

27.5 

3.7 

30.5 

0.3 


Data for students from ACT Class Profile Report (2004, 2005, 2006, and 2007). 


While females had higher median average ACTC seores and HSGPAs than males in the 
total applieant sample and higher median average HSGPA values for enrolled students, the 
median average ACTC seore for enrolled male and female students was the same. For the total 
applieant sample and among enrolled students, higher-ineome students tended to have the 
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highest median average ACTC seore and HSGPA values aeross institutions, followed by middle- 
ineome students, and then finally by lower-ineome students. Median ACTC seore and HSGPA 
were higher for White students in both student groups than for African American and Hispanic 
students."^ 

In the enrolled sample, a greater percentage of female students than male students, on 
average, had a FYGPA of either 2.5 or higher, or 3.0 or higher. As income increased, the typical 
percentage of students with a FYGPA of 2.5 or higher, or 3.0 or higher increased. Additionally, a 
greater percentage of White students than either African American or Hispanic students had a 
FYGPA of 2.5 or higher or 3.0 or higher. 

Method 

The present research used the methods developed by Sawyer (1996) for validating 
educational selection decisions; the method frames validity evidence in terms of probable 
outcomes, given the predictors and outcome criteria used. 

Two general types of hierarchical logistic regression models were estimated for 
predicting attainment of two successive levels of FYGPA. The first was a total-group regression 
model, consisting of ACTC score, HSGPA, or ACTC score and HSGPA used jointly. The 
second was a model consisting of a demographic indicator for racial/ethnic group, gender, or 
income level, ACTC score, HSGPA, or ACTC score and HSGPA used jointly, and the 
interaction between the demographic indicator(s) and the achievement predictors. Separate 
models were estimated for African American vs. White students and Hispanic vs. White 
students. 


The average ACTC score and HSGPA for African American students may have been higher for the total applicant 
sample than for the enrolled sample because of the relatively small median N count at the institutions sampled; the 
median number of African American students across institutions was 95. 
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Hierarchical models account for variability across colleges in order to draw correct 
conclusions about predictor-outcome relationships. In this study, we allowed the intercepts and 
slopes of the main effects to vary randomly across institutions. The slopes of the demographic 
indicator and achievement interactions were included as fixed effects only. 

The interaction between ACTC score and HSGPA in the total-group joint models for 
both FYGPA success levels were not statistically significantly different from zero (p~0.99 for 
both success levels). Therefore, we did not include an ACTC-by-HSGPA interaction in the 
models. In a study involving only four-year institutions. Sawyer (2013) found a large interaction 
effect. One likely reason for the different results is that in the present study, we examined both 
two-year and four-year institutions, rather than only four-year institutions. 

Differential Prediction 

Differential prediction occurs when students who have the same test scores, but belong to 
different population groups, have different probabilities of success. In this study, we examined 
differential prediction by comparing the estimated probability of attaining specific levels of 
FYGPA among different groups. 

In practice, when institutions make predictions about the success of their applicants, if 
they are admitted and enrolled, they may consider numerous factors including cognitive and non- 
cognitive factors. ACT does not advocate making college success predictions solely on the basis 
of a single measure, such as a test score. This paper examines the accuracy of predictions based 
on two predictors, HSGPA and the ACT Composite score, and on the incremental contribution of 
each to ameliorating over- and underprediction. The methods used here can also be generalized 
to multiple predictors. 
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Differential Validity 

One of the effeets of differential predietion is that if an institution used eutoff seores 
based on students’ probability of sueeess to make admission deeisions, different aggregate 
results eould result for different population groups. For example, predietive eorrelations eould 
differ among the groups. Or, the proportion of admitted applieants who are suceessful (sueeess 
rate) and the proportion of eorreet admission deeisions (aeeuraey rate), eould differ. We refer to 
sueh differenees as differential validity. 

Postseeondary institutions do not utilize striet seore eutoff values as those used in the 
present study. The use of striet eutoffs in the present study is a mathematieal idealization 
intended to provide guidanee to postseeondary institutions as they deeide how best to make 
admission deeisions. 

Validity statisties were generated from the hierarehieal logistie models and frequency 
distributions of ACTC score and HSGPA to determine the effectiveness of these measures for 
making postseeondary admission decisions. For each of the predictors investigated, alone or in 
combination, three validity statistics were calculated per institution using the institution specific 
total-group optimal cutoff : accuracy rate (AR), success rate (SR), and increase in accuracy rate 
(AAR). ^ 

For each institution and success level, optimal cutoffs that maximized prediction 
accuracy for FYGPA were identified for the ACTC score, HSGPA, and joint ACTC/HSGPA 
models using a total-group model. The cutoffs were used in order to simulate the effects of 
making admission decisions based on ACTC score and/or HSGPA on student subgroups. 

The accuracy rate is the estimated percentage of students for whom a correct admission 
decision is made. Accuracy rates are based on those students who would be admitted and be 
^ See Sawyer (2013) for methodologieal details. 
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successful as well as those students who would not be admitted and not have been sueeessful, 
had they been admitted, given the optimal cutoff. The sueeess rate is the estimated pereentage of 
students who, if enrolled based on some optimal cutoff, would be sueeessful. The inerease in 
aeeuraey rate or ineremental utility is the differenee between using the given eutoff for the 
predictor(s) to make admission deeisions and admitting all applicants (i.e. not using the 
predietor(s)). If no seleetion proeedure were used (i.e., if all students were admitted), a eertain 
pereentage would be sueeessful. This percentage is referred to as the “baseline” aeeuraey rate. 
The arithmetie differenee between the maximum aeeuraey rate and the baseline aeeuraey rate 
represents the inerease in aeeuraey rate (AAR) that results from using test seores and/or HSGPA 
for making admission deeisions. 

It can be shown that optimal eutoffs also eorrespond to a 0.50 probability of sueeess for a 
given model. For the ACTC seore and HSGPA joint model, multiple eombinations of ACTC 
seore and HSGPA eutoffs eorresponding to a probability of sueeess of 0.50 ean be identified. 
Probability distributions that cross 0.50 will yield aeeuraey rate distributions that increase to a 
maximum and then deerease. If the probability distribution for an institution does not eross 0.50, 
the maximum accuracy rate and optimal cutoff indicate that the seleetion eriteria are not useful, 
and the model is therefore eonsidered a “nonviable” model for an institution. Models for 
institutions with probability curves erossing 0.50 are referred to here as “viable” models. 

For eaeh model investigated, the number of institutions produeing viable models varied. 
The results presented are limited to institutions that produeed viable models for the three 
predietor models examined (i.e. ACTC seore, HSGPA, and joint ACTC seore and HSGPA 
models). In the 2.5 or higher and 3.0 or higher sueeess models, 253 and 247 institutions, 
respeetively, produeed viable models. 
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Total-group and subgroup validity statistics were based on the institution’s own 
frequeney distribution of predietor variables and summarized across institutions using median 
values. Results for eaeh model were based on using the institution specific total-group cutoffs 
and applying the eutoff to the subgroup-specific probability and frequency distribution for each 
institution. These values were used to compare subgroups in order to examine the differential 
usefulness in making admission decisions. Typical values of the validity statisties at the total- 
group optimal cutoffs were eompared aeross student subgroups. 

Results 

Total-group Results 

This section presents the median probabilities of suceess and validity statistics for the 
total-group analysis based on ACTC score and HSGPA values, alone and in combination, across 
institutions with viable models. Results for these models, whieh did not include student 
characteristics as predictors, serve as comparative baselines for evaluating the results presented 
for student subgroups. 

Table 5 ineludes the validity statistics that were ealculated for the two FYGPA levels, 
based on total-group models. Results presented inelude: number of institutions with viable 
models (N), optimal cutoffs (OC), maximum accuracy rate (AR), inerease in AR (AAR), success 
rate (SR), and observed pereentage of students below the optimal cutoff (PB). The joint usage of 
ACTC seore and HSGPA resulted in a higher median values for maximum AR, AAR, and SR for 
both suceess levels than using either predictor alone. ^ 


^ Readers interested in further information about the ineremental validity of ACTC seore and HSGPA are referred to 
Sawyer (2013). 
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Table 5 


Median Validity Statistics, Across Institutions, for Predicting Specific Levels of FYGPA 





Maximum 



Observed 




Accuracy 


Success 

percentage 




rate 

Increase in 

rate 

below OC 



Optimal 

cutoff 

(AR) 

AR (AAR) 

(SR) 

(PB) 



Median 

Median 

Median 

Median 

Predictor variable 

N 

(OC) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

2.5 or higher FYGPA 



18 

70 

8 

70 

30 

ACTC 


(57/91) 

(0/56) 

(57/91) 

(0/94) 

HSGPA 

253 

2.8 

72 

(57/91) 

9 

(0/52) 

72 

(51/91) 

29 

(0/95) 




73 

13 

74 

34 

ACTC & HSGPA 



(57/90) 

(0/56) 

(57/91) 

(0/86) 

3.0 or higher FYGPA 



23 

72 

31 

67 

70 

ACTC 


(63/92) 

(0/81) 

(52/93) 

(0/99) 

HSGPA 

247 

3.4 

73 

(56/94) 

31 

(0/65) 

65 

(51/96) 

63 

(0/98) 




76 

36 

69 

66 

ACTC & HSGPA 



(61/94) 

(0/82) 

(52/95) 

(0/99) 


Note: Multiple eombinations of ACTC seore and HSGPA eorrespond to a 0.50 probability of sueeess for the joint 


models. 


As the sueeess level inereased, the number of institutions with viable models deereased. 
The optimal cutoffs for both ACTC score and HSGPA also increased as the success level 
increased. 

Subgroup Differential Effects 

In this section the probabilities of success and validity statistics associated with using the 
total-group cutoff for predicting FYGPA are presented for each of the student demographic 
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subgroups/ Results are presented first for race/ethnicity, followed by gender, and finally income. 
For each subgroup, comparisons to the total-group probabilities are presented to illustrate further 
the differential effect of using a total-group cutoff for ACTC score, HSGPA, or both to predict 
FYGPA. 

Race/Ethnicity, For White, African American, and Hispanic students, as ACTC score or 
HSGPA increased, the probability of success also increased (Figure 1 and Figure 2). For the two 
FYGPA levels. White students had higher estimated probabilities of success than African 
American and Hispanic students over most of the ACTC score and HSGPA scales, and Hispanic 
students tended to have higher estimated chances of success than African American students. 



2.5 or higher (W hite) 2.5 or higher (Hispanic) 2.5 or higher (African American) 2.5 or higher (Total) 

^ ^ — 3.0 or higher (White) ^ ^ — 3.0 or higher (Hispanic) ^ ^ — 3.0 or higher (African American) " 3.0 or higher (Total) 


Note: The three vertieal reference lines represent the first, second, and third quartiles. 

Figure 1: Estimated probabilities of achieving specific FYGPA levels based on ACTC score, by 
race/ethnicity. 


^ Probabilities of success presented in these sections are based on the fixed effect parameter estimates from 
hierarchical logistic models. 
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2.5 or higher (White) 2.5 or higher (Hispanic) 2.5 or higher (African American) 2.5 or higher (Total) 

3.0 or higher (White) 3.0 or higher (Hispanic) 3.0 or higher (African American) • 3.0 or higher (Total) 


Note: The three vertieal reference lines represent the first, second, and third quartiles. 

Figure 2: Estimated Probabilities of FYGPA Achievement Based on HSGPA, by Race/ethnicity 

For both success levels, differences in probabilities between African American and White 
students tended to be of greater magnitude than the differences between Hispanic and White 
students at ACTC scores below 29. Differences in estimated probabilities between African 
American and White students scoring between the first and third ACTC score quartiles (17 and 
24, respectively) for the two FYGPA success levels were, at most, 0.16. Estimated probabilities 
of success for Hispanic students were, at most, 0.06 lower than those for White students. 

Conversely, as HSGPA increased, differences between estimated probabilities for White 
students and both African American and Hispanic students tended to increase. On average, 
estimated probabilities for African American and Hispanic students scoring between the first and 
third HSGPA quartile (about 2.8 and 3.8, respectively) for both success levels were, at most, 
0.21 lower than those of White students. For Hispanic students these probabilities were, at most. 
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0.08, lower than those of White students. The differenees between Afriean American and White 
students were larger in magnitude than those based on the ACTC score model, particularly at the 
3.0 or higher success level. 

Using ACTC score as the sole academic predictor of success, the difference in the 
estimated probabilities between the total-group model and the probabilities for White students 
generally decreased slightly as ACTC score increased. Additionally, as the success level 
increased, this underprediction decreased in magnitude for the same ACTC score. For African 
American and Hispanic students the under- and overprediction by the total-group tended to be 
smaller at the upper and lower extremes of the ACTC score scale and larger in the middle of the 
scale. In contrast to White students, as the success level increased, the magnitude of 
overprediction for African American and Hispanic students tended to increase slightly for similar 
ACTC scores. 

When HSGPA was used as the sole predictor of success, the differences in probabilities 
between the total-group model and White students were similar across levels, never exceeding a 
difference of 0.04. The differences in probability estimates for African American and White 
students in the 2.5 or higher success level tended to increase across much of the HSGPA scale. 
For African American students in the 3.0 or higher success level and for Hispanic students at 
both success levels, these differences tended to increase as HSGPA increased. 

Where differences in over- and underprediction of success existed, they tended to be of 
greater magnitude when HSGPA was used as the academic predictor then when ACTC score was 
used (see Figure 2). This was particularly notable for African American students scoring above a 
HSGPA of about 3.0. This suggested a total-group HSGPA model considerably overestimates 
the chances of success for African American and Hispanic students with a high HSGPA. 
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The median probabilities of suceess across institutions based on a total-group cutoff for 
racial/ethnic groups tended to show a pattern of underprediction for white students and over 
prediction for both Hispanic and African American students (see Table 6). Across institutions, 
for the 2.5 or higher success level, Hispanic students showed the least amount of overprediction. 
African American students, however, showed evidence of moderate overprediction. For the 3.0 
or higher success level, the overprediction observed for minority groups increased in magnitude. 
This was most dramatically seen for African American students. 
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Table 6 

Median Statistics for Predicting Specific Levels of FYGPA by Ethnicity Across Institutions 





Subgroup- 

Maximum 



Observed 




specific 

probability 

Accuracy 

Rate 

Increase in 

Success 

rate 

percentage 
below OC 


Total- 

group 


of success 

(AR) 

AR (AAR) 

(SR) 

(PB) 

Predictor 

Race/ 

Median 

Median 

Median 

Median 

Median 

Variable 

N Cutoff 

ethnicity 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

2.5 or higher FYGPA 



White 

0.56 

(0.29/0.77) 

69 

(52/97) 

5 

(-13/50) 

72 

(52/97) 

29 

(0/97) 

ACTC 

18 

African 

0.39 

70 

37 

52 

70 

American 

(0.19/0.61) 

(46/93) 

(0/86) 

(18/86) 

(0/100) 



Hispanic 

0.51 

(0.2/0.71) 

65 

(54/86) 

21 

(-13/72) 

59 

(26/84) 

70 

(0/100) 



White 

0.53 

(0.07/0.77) 

72 

(55/96) 

6 

(-4/50) 

74 

(51/96) 

25 

(0/91) 

HSGPA 

242 2.8 

African 

American 

0.35 

(0.07/0.61) 

67 

(33/90) 

29 

(-1/81) 

51 

(11/82) 

55 

(0/100) 



Hispanic 

0.47 

(0.23/0.7) 

67 

(42/84) 

19 

(-7/69) 

62 

(18/82) 

55 

(0/100) 



White 

0.52 

(0.1/0.73) 

73 

(59/97) 

10 

(-4/57) 

75 

(52/97) 

31 

(0/92) 

ACTC & 


African 

0.37 

73 

42 

55 

70 

HSGPA 


American 

(0.11/0.85) 

(45/94) 

(0/87) 

(10/86) 

(0/100) 



Hispanic 

0.48 

(0.2/0.73) 

70 

(55/87) 

31 

(-11/74) 

62 

(19/83) 

70 

(0/100) 
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Table 6 {continued) 





Subgroup- 

Maximum 



Observed 




specific 

probability 

Accuracy 

Rate 

Increase in 

Success 

rate 

percentage 
below OC 


Total- 

group 


of success 

(AR) 

AR (AAR) 

(SR) 

(PB) 

Predictor 

Race/ 

Median 

Median 

Median 

Median 

Median 

Variable 

N Cutoff 

ethnicity 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

3.0 or higher FYGPA 



White 

0.54 

(0.37/0.75) 

71 

(62/90) 

25 

(-2/63) 

68 

(53/90) 

66 

(1/99) 

ACTC 

23 

African 

0.36 

86 

71 

46 

93 

American 

(0.21/0.74) 

(57/97) 

(7/93) 

(7/75) 

(14/100) 



Hispanic 

0.45 

(0.32/0.6) 

78 

(60/91) 

56 

(2/83) 

53 

(16/77) 

93 

(14/100) 



White 

0.52 

(0.23/0.79) 

72 

(55/87) 

22 

(0/60) 

68 

(51/89) 

55 

(1/98) 

HSGPA 

236 3.4 

African 

American 

0.27 

(0.15/0.51) 

81 

(43/98) 

64 

(0/97) 

37 

(2/66) 

85 

(0/100) 



Hispanic 

0.42 

(0.21/0.59) 

75 

(53/96) 

49 

(0/92) 

52 

(4/77) 

85 

(0/100) 



White 

0.51 

(0.36/0.69) 

75 

(57/90) 

30 

(1/69) 

70 

(54/90) 

62 

(2/97) 

ACTC & 


African 

0.32 

87 

73 

48 

92 

HSGPA 


American 

(0.02/0.61) 

(61/100) 

(14/99) 

(1/87) 

(19/100) 



Hispanic 

0.43 

(0.04/0.6) 

81 

(63/98) 

61 

(6/96) 

55 

(3/80) 

93 

(18/100) 


Note: Multiple eombinations of ACTC seore and HSGPA eorrespond to a 0.50 probability of sueeess for the 
joint models. 


Table 6 displays the median validity statisties assoeiated with estimating FYGPA sueeess 
using ACTC seore and HSGPA individually and in eombination for White, Afriean Ameriean, 
and Hispanie students. The joint ACTC seore and HSGPA model tended to produee the most 
favorable ARs and SRs, on average aeross the raeial/ethnie groups. For the 2.5 or higher FYGPA 
sueeess level, White students generally had the highest median ARs and SRs. For the 3.0 or 







25 


higher FYGPA success level, median ARs were highest for African American students, and 
lowest for White students. 

Regardless of the success level or predictor model. White students had the highest typical 
SRs followed by Hispanic students and finally by African American students. For each 
racial/ethnic group as the success level increased, typical SRs decreased for all predictor models. 

For the 2.5 or higher FYGPA success level, African American students had the highest 
median AR when ACTC score was used followed by White and then Hispanic students. For the 
HSGPA model. White students had the highest median AR followed by both African American 
and Hispanic students. In the joint ACTC score and HSGPA model. White and African 
American students had the highest median AR followed by Hispanic students. Regardless of the 
predictor model, for the 2.5 or higher FYGPA success level. White students had considerably 
fewer students below the total-group optimal cutoff. Regardless of the predictor model, for the 
3.0 or higher success level, African American students had the highest median AR followed by 
Hispanic and then by White students. Additionally, regardless of predictor model. White students 
had fewer students below the total-group optimal cutoff followed by both African American and 
Hispanic students. In fact, for the 3.0 or higher success level, most African American and 
Hispanic students did not meet the total-group cutoff. 

Gender. For both males and females, as ACTC score or HSGPA increased, the estimated 
probability of attaining the two FYGPA success levels also increased (see Figure 3 and Figure 
4). Moreover, regardless of level of success examined, female students had a higher probability 
of success than male students. Differences in probabilities between males and females achieving 
a FYGPA of 2.5 or higher, or 3.0 or higher, were largest for students with ACTC scores of 21 - 
23 and 25 - 30, respectively. These differences in probabilities between males and females were. 
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at most, 0.17. Across the HSGPA scale, the differenees in probabilities between female and male 
students for the two success levels were, at most, 0.07. Differences in probabilities in both the 
ACTC seore and HSGPA models were larger for the 3.0 or higher model than for the 2.5 or 
higher model. Additionally, there appeared to be a trend of greater overprediction for males than 
underpredietion for females. 



2.5 or higher (Female) 2.5 or higher (Male) 2.5 or higher (Total) 

3.0 or higher (Female) 3.0 or higher (Male) — 3.0 or higher (Total) 

Note: The three vertieal reference lines represent the first, second, and third quartiles. 

Figure 3. Estimated probabilities of aehieving speeifie FYGPA levels based on ACTC seore, by 
gender. 
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3.0 or higher (Female) 3.0 or higher (Male) — 3.0 or higher (Total) 

Note: The three vertieal referenee lines represent the first, seeond, and third quartiles. 

Figure 4. Estimated probabilities of achieving specific FYGPA levels based on HSGPA, by 
gender. 

The estimated probability of success was higher for females and lower for males than the 
estimates produced by a total-group model over much of the ACTC score and HSGPA scales 
(see Figure 3 and Figure 4). At the 2"‘^, and quartiles of the ACTC score and HSGPA 
distributions, as indicated by the three vertical reference lines, the total-group models based on 
ACTC score and HSGPA underpredicts FYGPA success for female students at both success 
levels and overpredicts success for male students. As ACTC score increased, the amount of over- 
and underprediction for the gender probabilities of success tended to increase slightly in 
magnitude between ACTC scores of about 16-27 (maximum absolute difference in gender 
specific probabilities and total-group probabilities was about 0.10). The absolute differences 
between the total-group estimated probabilities and the gender-specific estimated probabilities 


28 


also tended to inerease as HSGPA inereased, with the largest absolute differenee being about 
0.05. 

The median subgroup-speeifie probabilities of sueeess aeross institutions were also 
ealeulated for eaeh sueeess level and predietor model (see Table 7). For both sueeess levels, 
using a total-group eutoff seore underpredicted the probability of success for females and 
overpredicted the probability of success for males. Across institutions, the use of ACTC score 
alone resulted in slightly larger differential prediction than when HSGPA was used in isolation. 
These findings are consistent with those previously discussed. 
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Table 7 

Median Statistics for Predicting Specific Levels of FYGPA by Gender Across Institutions 




Total- 

group 

Cutoff 


Subgroup- 
speeifie 
probability 
of sueeess 

Maximum 

Aeeuraey 

Rate 

(AR) 

Inerease in 
AR (AAR) 

Sueeess 

rate 

(SR) 

Observed 
pereentage 
below OC 
(PB) 

Predietor 

Variable 

N 

Gender 

Median 

(Min/Max) 

Median 

(Min/Max) 

Median 

(Min/Max) 

Median 

(Min/Max) 

Median 

(Min/Max) 

2.5 or higher FYGPA 




Female 

0.56 

73 

8 

75 

33 

ACTC 


18 

(0.40/0.68) 

(59/97) 

(0/64) 

(58/97) 

(0/96) 


Male 

0.45 

69 

16 

62 

40 




(0.33/0.61) 

(55/92) 

(0/73) 

(33/92) 

(0/100) 

HSGPA 

253 

2.8 

Female 

Male 

0.53 

(0.16/0.61) 

0.47 

73 

(57/96) 

70 

6 

(0/51) 

13 

75 

(53/96) 

66 

24 

(0/89) 

35 




(0.18/.059) 

(56/91) 

(-1/64) 

(48/91) 

(0/94) 

ACTC& 

HSGPA 



Female 

Male 

0.52 

(0.07/0.61) 

0.45 

(0.10/0.57) 

75 

(60/97) 

72 

(57/92) 

12 

(0/65) 

20 

(0/74) 

76 

(53/97) 

66 

(45/92) 

33 

(0/93) 

44 

(0/99) 

3.0 or higher FYGPA 




Female 

0.59 

74 

27 

74 

68 

ACTC 


23 

(0.36/0.73) 

(63/92) 

(0/76) 

(56/93) 

(0/99) 


Male 

0.43 

(0.32/0.57) 

74 

(59/94) 

43 

(0/89) 

58 

(20/91) 

74 

(0/100) 

HSGPA 

247 

3.4 

Female 

Male 

0.52 

(0.32/0.61) 

0.46 

(0.32/0.54) 

73 

(62/93) 

74 

(59/92) 

24 

(0/62) 

38 

(0/78) 

68 

(47/93) 

60 

(36/94) 

54 

(1/98) 

66 

(1/99) 

ACTC & 
HSGPA 



Female 

Male 

0.53 

(0.28/0.6) 

0.44 

(0.3/0.53) 

77 

(66/92) 

78 

(64/95) 

32 

(0/78) 

45 

(0/90) 

73 

(53/93) 

62 

(32/93) 

62 

(0/99) 

73 

(1/100) 


Note: Multiple eombinations of ACTC seore and HSGPA eorrespond to a 0.50 probability of sueeess for the joint 
models. 
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Table 7 further shows the median validity statisties aeross institutions for the ACTC 
seore, HSGPA, and joint ACTC seore and HSGPA models. For the 2.5 or higher sueeess level, 
using a total-group cutoff resulted in higher median ARs and SRs for female students than for 
male students, regardless of the predictor combination used. At the 3.0 or higher level, while the 
median SR was higher for females than for males, median ARs were more similar between males 
and females. For both success levels, typical AARs were considerably larger for males than for 
females, and a smaller percentage of males were at or above the total-group cutoff than were 
females. For both success levels the joint ACTC score and HSGPA model tended to produce 
more favorable ARs and SRs, on average for both males and females. 

Income, For lower-, middle-, and higher-income students, as ACTC score or HSGPA 
increased, the estimated probability of achieving the two FYGPA levels also increased (see 
Figure 5 and Figure 6). For both success levels, when either ACTC score or HSGPA was used as 
the sole academic predictor, the estimated probabilities of success for lower-income students 
tended to be lower than the estimated probabilities for middle-income students, and both tended 
to be lower than the estimated probabilities of higher-income students. 
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2.5 or higher (Higher-income) 2.5 or higher (Middie-income) — ‘ 2.5 or higher (Lower-income) 2.5 or higher (Total) 

— — — 3.0 or higher (Higher-income) — — — 3.0 or higher (Middie-income) — — — 3.0 or higher (Lower-income) « 3.0 or higher (Totai) 


Note: The three vertieal referenee lines represent the first, seeond, and third quartiles. 

Figure 5. Estimated probabilities of achieving specific FYGPA levels based on ACTC, by 
income. 



HSGPA 

2.5 or higher (Higher-income) 2.5 or higher (Middie-income) 2.5 or higher (Lower-income) 2.5 or higher (Total) 

3.0 or higher (Higher-income) 3.0 or higher (Middle-income) 3.0 or higher (Lower-income) • 3.0 or higher (Total) 


Note: The three vertieal referenee lines represent the first, seeond, and third quartiles. 

Figure 6. Estimated probabilities of FYGPA achievement based on HSGPA, by income. 
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In the two ACTC score models, the differences between income levels in estimated 
probabilities for students scoring between the first and third ACTC score quartile, averaged at 
most 0.07. As the success level examined increased from 2.5 or higher to 3.0 or higher, the 
difference in chances of attaining the success level between income levels decreased. Analogous 
differences between income groups, when HSGPA was used as the predictor, tended to be 
slightly larger in magnitude. This was most apparent in the differences in estimated probabilities 
between lower- and higher-income students. 

The probability estimates produced by the total-group model tended to be higher than the 
estimates for lower- income students for the two FYGPA success levels. The opposite tended to 
be true for higher- income students. For the two success levels, the differences in estimated 
probabilities for middle-income students and the total-group estimates when HSGPA or ACTC 
score was used as the sole academic predictor were, at most, about 0.04 and 0.03, respectively. 

Where over- and underprediction between the total-group and income specific 
probabilities existed, the differences tended to be smaller in magnitude when ACTC score was 
used particularly at higher levels of HSGPA and for the 3.0 or higher success level. This 
suggested that when ACTC score was used in the model rather than HSGPA over- and 
underprediction was reduced. 

The median probability of success at the total-group cutoff for lower- and higher-income 
students tended to be over- and underpredicted, respectively (see Table 8). Relatively little 
evidence of over- or underprediction was observed for middle-income students. These institution 
based estimates of differential prediction are consistent with the results based on the fixed effects 
of the hierarchical models previously presented. 
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Table 8 


Median Statistics for Predicting Specific Levels of FYGPA by Income Across Institutions 






Subgroup- 

Maximum 



Observed 





specific 

probability 

Accuracy 

Rate 

Increase in 

Success 

rate 

percentage 
below OC 



Total- 

group 


of success 

(AR) 

AR (AAR) 

(SR) 

(PB) 

Predictor 



Median 

Median 

Median 

Median 

Median 

Variable 

N 

Cutoff 

Income 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

2.5 or higher FYGPA 




Lower 

0.49 

69 

18 

63 

50 




(0.44/0.53) 

(53/90) 

(0/81) 

(48/90) 

(0/99) 

ACTC 


18 

Middle 

0.52 

(0.48/0.57) 

70 

(55/95) 

10 

(0/76) 

70 

(54/95) 

37 

(0/98) 




Higher 

0.55 

(0.5/0.61) 

71 

(55/97) 

5 

(0/67) 

73 

(57/97) 

27 

(0/98) 




Lower 

0.47 

68 

12 

63 

36 




(0.39/0.54) 

(51/90) 

(-1/60) 

(45/90) 

(0/95) 

HSGPA 

253 

2.8 

Middle 

0.49 

(0.34/0.55) 

72 

(55/95) 

10 

(0/52) 

72 

(54/95) 

29 

(0/90) 




Higher 

0.53 

(0.37/0.62) 

74 

(57/96) 

6 

(0/41) 

77 

(56/96) 

24 

(0/90) 




Lower 

0.47 

(0.09/0.59) 

72 

(51/90) 

22 

(0/79) 

65 

(49/91) 

49 

(0/99) 

ACTC& 



Middle 

0.50 

74 

14 

73 

36 

HSGPA 



(0.1/0.54) 

(55/96) 

(0/75) 

(54/96) 

(0/99) 




Higher 

0.53 

(0.08/0.59) 

75 

(56/97) 

10 

(0/66) 

77 

(55/97) 

30 

(0/98) 
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Table 8 {continued) 






Subgroup- 

Maximum 



Observed 





specific 

probability 

Accuracy 

Rate 

Increase in 

Success 

rate 

percentage 
below OC 



Total- 

group 


of success 

(AR) 

AR (AAR) 

(SR) 

(PB) 

Predictor 



Median 

Median 

Median 

Median 

Median 

Variable 

N 

Cutoff 

Income 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

(Min/Max) 

3.0 or higher FYGPA 




Lower 

0.48 

76 

46 

61 

81 




(0.43/0.53) 

(60/92) 

(0/83) 

(41/86) 

(0/100) 

ACTC 


23 

Middle 

0.52 

(0.48/0.6) 

74 

(63/92) 

33 

(0/77) 

67 

(52/93) 

71 

(0/99) 




Higher 

0.54 

(0.5/0.65) 

72 

(61/96) 

24 

(0/68) 

69 

(54/96) 

63 

(0/97) 




Lower 

0.43 

72 

38 

54 

68 




(0.35/0.59) 

(53/89) 

(0/70) 

(39/90) 

(2/99) 

HSGPA 

247 

3.4 

Middle 

0.49 

(0.46/0.58) 

74 

(58/94) 

29 

(0/66) 

65 

(52/95) 

58 

(2/98) 




Higher 

0.54 

(0.47/0.64) 

73 

(58/97) 

22 

(0/61) 

70 

(56/97) 

54 

(1/97) 




Lower 

0.45 

(0.37/0.56) 

77 

(60/92) 

47 

(0/84) 

61 

(44/90) 

76 

(0/99) 

ACTC & 



Middle 

0.50 

77 

36 

69 

66 

HSGPA 



(0.44/0.55) 

(62/93) 

(0/78) 

(51/95) 

(0/99) 




Higher 

0.53 

(0.38/0.69) 

76 

(59/95) 

28 

(0/69) 

72 

(59/97) 

60 

(0/96) 


Note: Multiple combinations of ACTC score and HSGPA correspond to a 0.50 probability of success for the joint 
models. 


Table 8 shows the median validity statisties using ACTC seore and HSGPA both 
independently and jointly to prediet FYGPA sueeess levels using the total-group eutoff values 
aeross ineome levels and institutions. At the 2.5 or higher FYGPA sueeess level, as ineome level 
inereased, typieal ARs also inereased slightly. For both sueeess levels, as ineome inereased, 
median AARs, as well as the observed pereentage of students below the total-group eutoff, 
tended to deerease and median SRs inereased. For the 3.0 or higher level, as ineome inereased. 
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typical ARs tended to deerease. Median ARs for lower- and middle-ineome students were higher 
for the 3.0 or higher level followed by the 2.5 or higher sueeess level; median ARs were 
eomparable between the two sueeess levels. The typieal SRs for eaeh ineome level tended to 
deerease as the FYGPA sueeess level inereased. For both sueeess levels the joint ACTC seore 
and HSGPA model tended to produee slightly more favorable ARs and SRs, on average, aeross 
the ineome groups. 

Conclusions 

The raee/ethnieity models show that for the ACTC seore and HSGPA models the total- 
group model tends to underestimate FYGPA sueeess for White students, while overestimating 
sueeess for Afriean Ameriean and Hispanie students. For both the ACTC seore and HSGPA 
models at both sueeess levels. White students have the highest estimated ehanees of sueeess, 
typieally followed by Hispanie students and then by Afriean Ameriean students. For both sueeess 
levels, differenees in the estimated ehanees of sueeess between the three raee/ethnieity groups 
deerease as ACTC seore inereases. This is not true for the HSGPA models where the differenees 
tend to persist. 

In the HSGPA models, the ehanees of sueeess for Afriean Ameriean and Hispanie 
students are overestimated, partieularly at higher levels. As a result, more students with a high 
HSGPA may be admitted to an institution and subsequently fail to attain higher levels of 
FYGPA. 

Both the ACTC seore and HSGPA total-group models underestimate the ehanees of 
sueeess for females and overestimate them for males. Using a total-group eutoff to make 
admission deeisions resulted in slightly greater predietion aeeuraey for females than for males at 
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the 2.5 or higher success level, but the percentage of correct classifications were more 
comparable for females and males at the 3.0 or higher success level. 

At the 2.5 or higher and 3.0 or higher levels, the joint model tends to produce greater 
prediction accuracy then when either ACTC score or HSGPA is used alone. Therefore, the use of 
a joint model which takes into consideration both ACTC score and HSGPA results in greater 
ability to accurately make admission decisions. Additionally, in the 2.5 or higher and 3.0 or 
higher success levels, the joint model tends to produce higher median ARs and SRs than when 
either predictor is used alone. At these levels the joint model tends to result in greater prediction 
accuracy than when either predictor is used alone. 

When income is examined, in both the ACTC score and HSGPA models, the total-group 
model tends to underestimate the chances of success of higher- income students and overestimate 
the chances of lower-income students. For the HSGPA models, the estimates of success for 
middle-income students and the estimates from the total-group model do not differ substantially. 
For the ACTC score models, however, the total-group model tends to slightly underestimate the 
chances of success for middle-income students. Using the total-group cutoff at the 2.5 or higher 
level, as income increases, prediction accuracy increases in the ACTC score, HSGPA, and joint 
models. At the 3.0 or higher level, for the ACTC score, HSGPA, and joint models, as income 
increases, correct classifications remain similar while student success at meeting the FYGPA 
success level increases. 

These findings are generally consistent with Sawyer (2013) who found that the joint use 
of ACTC score and HSGPA for making admission decisions, regardless of subgroup 
membership, resulted in greater incremental accuracy rates than when either predictor is used 
alone for a cutoff proportion of 0.50. Furthermore the current study found, as Sawyer (2013) did. 
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that the joint usage of both predietors was moderately effeetive at inereasing the incremental 
success rates than when either predictor was used alone. 

The current study also provides evidence that the findings of Noble and Sawyer (2002) of 
using ACTC score and HSGPA jointly for making admission decisions resulted in greater 
prediction accuracy than using either predictor alone, regardless of subgroup membership, 
generally hold for racial/ethnic, gender, and income subgroups at the success levels of 2.5 or 
higher or 3.0 or higher FYGPA. 

Implications 

We examined the differential effect of using ACTC score, HSGPA, and both ACTC 
score and HSGPA jointly to predict FYGPA and make college admission decisions for student 
subgroups. The models developed provide insight into the use of the college readiness indicators 
of ACTC score and HSGPA to identify applicants who have a good chance of succeeding in 
college. 

The results suggest that while subgroup differences exist, African American, Hispanic, 
and lower-income students are actually advantaged by the use of a total-group cutoff for making 
admission decisions. Additionally, it is worthwhile to note that the pattern of overprediction 
found for ACTC score is also seen for HSGPA. In fact, for income and race/ethnicity, as the 
success level increases, the differences in probability estimates between subgroups are larger for 
HSGPA than for ACTC score. Furthermore, the use of both ACTC score and HSGPA typically 
results in greater accuracy for predicting FYGPA success. 

The current findings suggest that both ACTC score and HSGPA underpredict the 
probability of female success. Additionally the differences in probabilities between males and 
females, as well as between each subgroup and the total-group probability of success, are larger 
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for ACTC score than for HSGPA. That said, however, when we examine the effects of using 
these predietors in isolation and in eombination for admission deeisions for males and females, 
the joint use of these predictors results in higher predietion aeeuraey at the sueeess levels of 2.5 
or higher and 3.0 or higher FYGPA. 

Additionally, in eomparing the estimated probability funetions for ACTC seore and 
HSGPA, we ean see two important differenees. First, it is apparent that the estimated probability 
funetion for HSGPA is shifted to the higher end of its seale. Praotieally speaking, this results in 
two not ineonsequential eoneems. While ACTC seore produees aetionable probability estimates 
over the entirety of its seale, students must seore in the upper half of the HSGPA seale in order to 
produee probability estimates that allow greater applieant differentiation. For example in order 
for students to have a 50% chanee of attaining a 2.5 or higher, or a 3.0 or higher, FYGPA a 
student would need to seore a 17 or 22 ACTC seore, respeetively, or a 2.85 or 3.41 HSGPA, 
respeetively. 

Second, while the estimated probability of sueeess at both success levels for students with 
an ACTC seore of 36 (the ACTC eeiling seore) approaeh 1.0, the probabilities for the HSGPA 
models do not. As a result students with higher HSGPAs are not estimated to have as high of a 
probability of sueeess as when ACTC seore is used. This finding is eonsistent with Sawyer 
(2013) who found that even for students with a high HSGPA the probability of a 3.0 or higher 
FYGPA depended on a student’s ACTC seore. 

As Noble and Sawyer (2002) found previously, HSGPA is problematic as a predictor of 
high levels of FYGPA. The present researeh suggests that HSGPA may funetion better as a 
predictor of lower levels of FYGPA than of higher levels of FYGPA. 
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The differences in the predicted probability of success that we found among groups might 
be due to differences in variables not included in the prediction models. For example, students in 
different groups might differ in their courses taken, in their psychosocial characteristics (e.g., 
self-efficacy, academic discipline), or in situational variables (e.g., number of dependents, 
number of hours worked at a job). 

These results do not support the claim that ACTC score adds little information to 
predicting college success outcomes. Instead, both ACTC score and HSGPA have an important 
role to play in helping college admission offices make informed admission decisions. Regardless 
of individual predictor performance, the present research supports using ACTC score and 
HSGPA jointly for measuring academic preparation and in making postsecondary admission 
decisions for all students. 
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